Skip to content

[WC-3334] Fix DG2 export type bugs#2182

Open
samuelreichert wants to merge 11 commits into
mainfrom
fix/datagrid-export-type-bugs
Open

[WC-3334] Fix DG2 export type bugs#2182
samuelreichert wants to merge 11 commits into
mainfrom
fix/datagrid-export-type-bugs

Conversation

@samuelreichert
Copy link
Copy Markdown
Contributor

Pull request type

Refactoring (e.g. file rename, variable rename, etc.)
Bug fix (non-breaking change which fixes an issue)

Description

Fix multiple data export bugs in DataGrid2's Excel export feature (WC-3334).

Problems

  1. customContent columns ignore exportType — exportValue always exports as text regardless of exportType setting, making numeric values unusable for calculations in Excel
  2. Date exports leak time components — Date-only formats (e.g., dd-MMM-yyyy) carry a hidden time component in the cell value, affecting sorting and calculations
  3. Boolean exports show TRUE/FALSE — Excel boolean cells always render as TRUE/FALSE regardless of the w field; DataGrid2 displays Yes/No in the UI
  4. Long numbers lose precision — Big.toNumber() loses precision for values >15 significant digits, causing scientific notation in Excel

Changes

  • Extracted cell-creation logic from DSExportRequest.ts into standalone cell-readers.ts module (pure functions, zero behavior change)
  • customContent reader now branches on exportType: parses to Number() for "number", new Date() for "date", with graceful string
    fallback on parse failure
  • Added hasTimeComponent(format) / stripTime(date) helpers — when format has no h/s tokens, date values are normalized to midnight UTC
  • excelBoolean now returns { t: "s", v: "Yes" | "No" } instead of { t: "b" } to match grid display
  • Big values with >15 significant digits export as strings via toFixed() to preserve all digits
  • 28 new unit tests covering all reader paths, edge cases, and fallback behavior

@samuelreichert samuelreichert marked this pull request as ready for review April 20, 2026 14:06
@samuelreichert samuelreichert requested a review from a team as a code owner April 20, 2026 14:06
@samuelreichert samuelreichert force-pushed the fix/datagrid-export-type-bugs branch 2 times, most recently from b804adb to 05d8784 Compare April 24, 2026 13:57
@samuelreichert samuelreichert force-pushed the fix/datagrid-export-type-bugs branch from 05d8784 to bc6c7a0 Compare April 30, 2026 07:52
@samuelreichert samuelreichert requested a review from iobuhov May 1, 2026 12:36
@samuelreichert samuelreichert force-pushed the fix/datagrid-export-type-bugs branch 3 times, most recently from 08b3fd2 to 66ebfc8 Compare May 7, 2026 14:02
iobuhov
iobuhov previously approved these changes May 7, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 7, 2026

AI Code Review

🔶 Changes requested — one or more medium-severity items must be addressed


What was reviewed

File Change
packages/pluggableWidgets/datagrid-web/src/features/data-export/cell-readers.ts New module: extracted cell-reader logic from DSExportRequest, added customContent type-branching, boolean→Yes/No, date time-stripping, Big precision guard
packages/pluggableWidgets/datagrid-web/src/features/data-export/DSExportRequest.ts Removed inline reader logic, imports readChunk from new module
packages/pluggableWidgets/datagrid-web/src/features/data-export/__tests__/cell-readers.spec.ts New test file: 28 tests covering reader paths, fallbacks, precision, and time-stripping
packages/pluggableWidgets/datagrid-web/CHANGELOG.md Added four ### Fixed entries under [Unreleased]

Skipped (out of scope): dist/, pnpm-lock.yaml


Findings

🔶 Medium — excelDate type signature allows {t:"d", v: string} — invalid SheetJS cell

File: packages/pluggableWidgets/datagrid-web/src/features/data-export/cell-readers.ts line 71
Problem: excelDate accepts value: string | Date and format?: string. When a caller passes a string value with a non-undefined format, the function returns { t: "d", v: "<string>" } — an invalid SheetJS cell (SheetJS requires t:"d" cells to have a Date as v). The current callers happen not to hit this combination, but the type signature doesn't prevent it. A future caller could easily produce a corrupt export.
Fix: Overload or split the function so t:"d" is only produced when v is a Date:

export function excelDate(value: string): ExcelCell;
export function excelDate(value: Date, format: string): ExcelCell;
export function excelDate(value: string | Date, format?: string): ExcelCell {
    return {
        t: format === undefined || typeof value === "string" ? "s" : "d",
        v: value,
        z: format
    };
}

🔶 Medium — dynamicText reader computes format but never uses it for numeric/date coercion

File: packages/pluggableWidgets/datagrid-web/src/features/data-export/cell-readers.ts line 142
Problem: The dynamicText reader calls getCellFormat(...) and passes format to excelString. However, excelString only uses format to set the z field — there is no coercion to t:"n" or t:"d" regardless of exportType. This means a dynamicText column with exportType: "number" still exports as a string cell, which is the bug described in problem #1 of the PR for customContent. The same silent no-op exists here but is untested and undocumented.
Fix: Either apply the same number/date branching as customContent, or explicitly document and test that dynamicText always exports as string (and why). If the intent is string-only, remove the getCellFormat call to avoid confusion.


⚠️ Low — hasTimeComponent regex misses minute-only formats like "mm:ss"

File: packages/pluggableWidgets/datagrid-web/src/features/data-export/cell-readers.ts line 86
Note: /[hs]/i detects h (hours) and s (seconds). A format like "mm:ss" (minutes and seconds, no hours) would match s and correctly preserve time — that path is fine. However, a format of "HH:mm" (hours and minutes, locale variants) would also match H via the case-insensitive flag — also correct. The one edge case not covered by a test is "ss" alone (elapsed seconds format) — worth adding a test case for documentation. No code change required if the current behavior is intentional.


⚠️ Low — excelString has redundant ?? undefined

File: packages/pluggableWidgets/datagrid-web/src/features/data-export/cell-readers.ts line 67
Note: z: format ?? undefined — since format is already string | undefined, this is a no-op. Simplify to z: format.


⚠️ Low — Plain number type path in attribute reader is untested

File: packages/pluggableWidgets/datagrid-web/src/features/data-export/__tests__/cell-readers.spec.ts
Note: The attribute reader at line 131 has a branch for typeof value === "number" (native JS number, not Big). All tests use new Big(...). Since Mendix's ListAttributeValue for numeric types always returns Big, this branch is likely dead code — if so, remove it. If it can be reached, add a test with a plain number builder or a comment explaining when it applies.


Positives

  • Clean extraction into a pure-function module (cell-readers.ts) with zero side effects — easy to test and reason about.
  • The countSignificantDigits guard for Big precision handles leading zeros, decimals, and negative values correctly.
  • stripTime uses Date.UTC consistently, avoiding local-timezone contamination in exported date values.
  • Boolean fix (Yes/No string cell instead of {t:"b"}) matches grid display and avoids the Excel TRUE/FALSE rendering issue cleanly.
  • Test structure uses the column() factory helper from test-utils correctly, keeping tests concise and avoiding repetition.
  • CHANGELOG entries are clear, user-facing, and cover all four fixed bugs.

@samuelreichert samuelreichert force-pushed the fix/datagrid-export-type-bugs branch 2 times, most recently from 4e8b769 to f494395 Compare May 12, 2026 14:10
@samuelreichert samuelreichert requested a review from iobuhov May 12, 2026 14:11
@github-actions
Copy link
Copy Markdown

AI Code Review

⚠️ Approved with suggestions — low-severity items only, safe to merge


What was reviewed

File Change
packages/pluggableWidgets/datagrid-web/CHANGELOG.md Four new Fixed entries under Unreleased
packages/pluggableWidgets/datagrid-web/src/features/data-export/DSExportRequest.ts Removed all inline cell-creation helpers; imports readChunk, HeaderDefinition, RowData from new module
packages/pluggableWidgets/datagrid-web/src/features/data-export/cell-readers.ts New module with all cell-reader logic, exported helpers, and readChunk
packages/pluggableWidgets/datagrid-web/src/features/data-export/__tests__/cell-readers.spec.ts 28 new unit tests covering all reader paths

Skipped (out of scope): dist/, pnpm-lock.yaml


Findings

⚠️ Low — hasTimeComponent regex misses minutes token m

File: packages/pluggableWidgets/datagrid-web/src/features/data-export/cell-readers.ts line 89
Note: The regex /[hs]/i matches h (hours) and s (seconds) but not m (minutes). A format like "hh:mm" would correctly match on h, but a pure "mm:ss" (minutes + seconds without an hour component) would not match h — it would still match s, so that edge case is covered. However, a format like "HH:MM" (uppercase) in a custom SheetJS format where only M = minutes is used would be missed entirely. In practice, any real time format includes h/H, so this is unlikely to cause a real regression, but the regex is under-documented.
Fix: Add a comment explaining which tokens are checked, or broaden to /[hms]/i to be safe:

// h = hours, m = minutes, s = seconds
function hasTimeComponent(format: string): boolean {
    return /[hms]/i.test(format);
}

⚠️ Low — exportValue read without checking .status

File: packages/pluggableWidgets/datagrid-web/src/features/data-export/cell-readers.ts line 157
Note: props.exportValue?.get(item).value ?? "" accesses .value without first checking .status === "available". If the value is in "loading" state, .value may be undefined (hence the ?? ""), but this silently emits an empty cell rather than signalling the loading state. The attribute and dynamicText readers both gate on .status; customContent does not. This was the same in the original code, so it's not a regression introduced here, but since the module is being rewritten it's a good opportunity to align.
Fix:

customContent(item, props) {
    const raw = props.exportValue?.get(item);
    if (!raw || raw.status !== "available") {
        return makeEmptyCell();
    }
    const value = raw.value ?? "";
    // ... rest of reader
}

⚠️ Low — excelDate overloads allow Date without format to slip through at runtime

File: packages/pluggableWidgets/datagrid-web/src/features/data-export/cell-readers.ts lines 71–79
Note: The two overload signatures enforce that Date requires a format argument at compile time, but the runtime implementation t: value instanceof Date && format !== undefined ? "d" : "s" silently falls back to "s" if a Date is passed without a format (which TypeScript should prevent, but any-typed call sites bypass this). The original code had the same shape; no regression introduced here, but since this is a clean extraction it's worth tightening.
Fix: Add a runtime guard or a non-null assertion with a comment, to make the intent explicit and catch unexpected call sites:

export function excelDate(value: string | Date, format?: string): ExcelCell {
    if (value instanceof Date && format === undefined) {
        // Should not happen — TypeScript overloads require format for Date
        return makeEmptyCell();
    }
    return {
        t: value instanceof Date ? "d" : "s",
        v: value,
        z: format
    };
}

Positives

  • Clean separation of concerns: extracting all cell-creation logic into a pure-function module (cell-readers.ts) with zero side effects makes the code straightforward to test and reason about in isolation.
  • Test coverage is thorough — all four reader types, edge cases (NaN, empty strings, parse failures), and the new time-stripping logic are all explicitly exercised using @mendix/widget-plugin-test-utils builders rather than manual mock objects.
  • The countSignificantDigits approach using Big.toFixed() + string manipulation is correct and avoids floating-point representation issues that value.toNumber() would introduce for big integers.
  • CHANGELOG entries are clear, user-facing, and follow the Keep a Changelog format already established in the file.

samuelreichert and others added 11 commits May 13, 2026 15:48
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Documents current behavior of attribute, dynamicText, and customContent
readers before bug-fix changes are applied.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…number

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…digits

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove "b" from ExcelCell.t union and boolean from ExcelCell.v
since excelBoolean now returns string cells. Fix misleading test
name for undefined dynamicText case.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add overloads to excelDate so t:"d" is only produced when v is a Date,
preventing invalid SheetJS cells. Remove dead plain-number branch in
attribute reader (Mendix always returns Big for numeric types). Drop
no-op getCellFormat call in dynamicText reader (pre-rendered strings
have no raw typed value to coerce). Remove redundant `?? undefined`
in excelString.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@samuelreichert samuelreichert force-pushed the fix/datagrid-export-type-bugs branch from f494395 to 5aef9f2 Compare May 13, 2026 13:49
@github-actions
Copy link
Copy Markdown

AI Code Review

🔶 Changes requested — one or more medium-severity items must be addressed


What was reviewed

File Change
packages/pluggableWidgets/datagrid-web/src/features/data-export/cell-readers.ts New module — extracted cell-creation logic with bug fixes for customContent exportType, date time-stripping, boolean display, and Big precision
packages/pluggableWidgets/datagrid-web/src/features/data-export/DSExportRequest.ts Deleted old in-file readers; now imports readChunk + types from cell-readers.ts
packages/pluggableWidgets/datagrid-web/src/features/data-export/__tests__/cell-readers.spec.ts 28 new unit tests covering all reader paths
packages/pluggableWidgets/datagrid-web/CHANGELOG.md Four entries added under ### Fixed in [Unreleased]

Skipped (out of scope): dist/, pnpm-lock.yaml


Findings

🔶 Medium — Whitespace-only strings silently export as the number 0

File: packages/pluggableWidgets/datagrid-web/src/features/data-export/cell-readers.ts line 164–167
Problem: Number(" ") evaluates to 0, not NaN. So when a user's exportValue expression returns a whitespace-only string and exportType is "number", the guard value !== "" passes, !Number.isNaN(0) is true, and the cell is exported as the number 0 — silently discarding the actual content. This differs from the intent of the fallback documented in the PR description.
Fix:

if (props.exportType === "number" && value.trim() !== "") {
    const parsed = Number(value);
    if (!Number.isNaN(parsed)) {
        return excelNumber(parsed, format);
    }
}

A companion test:

it("falls back to string for whitespace-only value with number exportType", () => {
    const col = column("Ws", c => {
        c.showContentAs = "customContent";
        c.exportValue = listExpression(() => "   ");
        c.exportType = "number";
    });
    const cell = readSingleCell(col);
    expect(cell.t).toBe("s");
    expect(cell.v).toBe("   ");
});

⚠️ Low — hasTimeComponent regex matches the S in locale specifiers

File: packages/pluggableWidgets/datagrid-web/src/features/data-export/cell-readers.ts line 89
Note: /[hs]/i (case-insensitive) matches any h, H, s, or S in the format string. A locale-tagged format like "[$-en-US]mmmm d, yyyy" contains a capital S in US, so hasTimeComponent incorrectly returns true and the time is not stripped — even though the format is date-only. Locale-tagged formats are uncommon in this context, but worth hardening. One approach: strip bracket-delimited locale tokens before testing, or use a stricter token regex:

function hasTimeComponent(format: string): boolean {
    // Strip locale tags like [$-en-US] before checking for time tokens
    const stripped = format.replace(/\[.*?\]/g, "");
    return /[hs]/i.test(stripped);
}

⚠️ Low — typeof value === "number" branch was removed without a comment

File: packages/pluggableWidgets/datagrid-web/src/features/data-export/cell-readers.ts line 133
Note: The old attribute reader handled both value instanceof Big and typeof value === "number". The new code only handles Big. If a Mendix attribute ever surfaces a plain JS number (e.g., an integer constant from a calculated attribute), it falls through to excelString(data.displayValue ?? "") — exporting a number as text without any warning. If this branch was intentionally dropped because Mendix guarantees numeric attributes are always Big, add a brief comment or an exhaustive check to make that contract explicit.


⚠️ Low — Missing test for dynamicText reader unavailable path

File: packages/pluggableWidgets/datagrid-web/src/features/data-export/__tests__/cell-readers.spec.ts
Note: The dynamicText reader returns excelString("n/a") when data.status === "unavailable" (line 150 of cell-readers.ts), but there is no test covering this branch. Since "n/a" is a meaningful user-visible fallback, add a test:

it("returns n/a cell when dynamicText is unavailable", () => {
    const col = column("Label", c => {
        c.showContentAs = "dynamicText";
        c.dynamicText = listExpression(() => "text", "unavailable");  // or however the builder sets unavailable
    });
    const cell = readSingleCell(col);
    expect(cell.t).toBe("s");
    expect(cell.v).toBe("n/a");
});

⚠️ Low — ### Fixed placed before ### Added in CHANGELOG

File: packages/pluggableWidgets/datagrid-web/CHANGELOG.md
Note: Keep a Changelog specifies section order as Added → Changed → Deprecated → Removed → Fixed → Security. The new entries put ### Fixed above the existing ### Added block. Not blocking, but worth reordering for consistency.


Positives

  • Clean extraction of readers into a dedicated cell-readers.ts module — pure functions, no side-effects, easy to unit-test in isolation.
  • Overloaded excelDate signatures enforce at the type level that a Date value requires a format argument, preventing accidental date-as-string cells.
  • countSignificantDigits correctly handles negatives, decimals, and leading zeros via progressive .replace chaining rather than fragile regex.
  • The value !== "" guard before Number(value) in customContent prevents the known Number("") === 0 false positive — the whitespace edge case is the only remaining gap.
  • 28 tests cover all three reader paths plus edge cases (loading, unavailable, precision overflow, time-stripping, fallback on parse failure) — thorough coverage for a bug-fix PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants